Hardware Accelerators Research Articles

VerSA: Versatile Systolic Array Architecture for Sparse and Dense Matrix Multiplications

A key part of modern deep neural network (DNN) applications is matrix multiplication. As DNN applications are becoming more diverse, there is a need for both dense and sparse matrix multiplications to be accelerated by hardware. However, most hardware accelerators are designed to accelerate either dense or sparse matrix multiplication. In this paper, we propose VerSA, a versatile systolic array architecture for both dense and sparse matrix multiplications. VerSA employs intermediate paths and SRAM buffers between the rows of the systolic array (SA), thereby enabling an early termination in sparse matrix multiplication with a negligible performance overhead when running dense matrix multiplication. When running sparse matrix multiplication, 256 × 256 VerSA brings performance (i.e., an inverse of execution time) improvement and energy saving by 1.21×–1.60× and 7.5–30.2%, respectively, when compared to the conventional SA. When running dense matrix multiplication, VerSA results in only a 0.52% performance overhead compared to the conventional SA.

A Flexible Hardware Accelerator for Booth Polynomial Multiplier

This article presents a parameterized/flexible hardware accelerator design tailored for the Booth polynomial multiplication method. The flexibility is achieved by allowing users to compute multiplication operations across various operand lengths, reaching up to 212 or 4096 bits. Our optimization strategy involves resource reuse, effectively minimizing the overall area cost of the Booth accelerator design. A comprehensive evaluation compares the proposed multiplier design with several non-digitized bit-serial polynomial multiplication accelerators. Implementation is realized in Verilog HDL using the Vivado IDE tool, featuring diverse operand sizes, and post-place and route assessments are performed on the Xilinx Virtex-7 field-programmable gate array device. For the largest considered operand size of 1024 × 1024, our Booth accelerator utilizes 1434 slices and can operate on a maximum frequency of 523.56 MHz. A single polynomial multiplication operation requires 0.977 μs and the total power consumption is 927 mW. Moreover, a comparison to state-of-the-art accelerators reveals that the proposed flexible accelerator is 1.34× faster in computation time and 1.05× more area-efficient than the recent dedicated polynomial multiplication design. Therefore, the implementation results and comparison to the state of the art show that the proposed accelerator is suitable for a wide range of cryptographic applications.

High-Speed Power Allocation in NOMA System Using FPGA-Based DNN

Artificial Intelligence (AI) is rapidly transforming the healthcare, finance and transportation industries. This paper presents a field-programmable gate array (FPGA)-based neural network accelerator (NNA) design for power allocation in downlink nonorthogonal multiple access (NOMA) networks. The proposed hardware accelerator effectively cuts computational costs while delivering performance on par with the highest sum capacity. Numerical results show that this NNA offers a remarkable computational speed increase of up to 99% compared to the conventional exhaustive search method. Furthermore, the deep learning (DL) model achieved high accuracy (0.92 training, 0.93 testing), and the hardware accelerator design for this DL inference model was implemented on the PYNQ-Z2 board-constrained edge device to predict power allocation coefficients in NOMA systems.

Vehicle Surroundings Perception Using Micro‐electromechanical Systems Inertial Sensors

Sensors enable vehicles to perceive their surroundings even while parked. Camera‐based systems for vehicle surroundings perception are already available as of today. However, camera‐based systems result in excessive power consumption and data collection. In this article, a study on always‐on surroundings perception of parked vehicles is done using micro‐electromechanical systems (MEMS) inertial sensors as an alternative to camera‐based systems. The measurements of the MEMS inertial sensors are used to detect and classify 17 different types of events. These events include accidents, vandalism, rain, as well as normal use. A convolutional neural network (CNN) is used to classify the events. The data used to train the CNN is recorded using a test vehicle in a laboratory setting and on a test track. The CNN is quantized for deployment on a CNN hardware accelerator which is packaged with the sensor as a system‐in‐package (SiP). The novel sensor system achieves a classification accuracy of 88% at a total system power dissipation of 27 μW.

Towards practical superconducting accelerators for machine learning using U-SFQ

Most popular superconducting circuits operate on information carried by ps-wide, \(\boldsymbol{\mu}\) V-tall, single flux quantum (SFQ) pulses. These circuits can operate at frequencies of hundreds of GHz with orders of magnitude lower switching energy than complementary-metal-oxide-semiconductors (CMOS). However, under the stringent area constraints of modern superconductor technologies, fully-fledged, CMOS-inspired superconducting architectures cannot be fabricated at large scales. Unary SFQ (U-SFQ) is an alternative computing paradigm that can address these area constraints. In U-SFQ, information is mapped to a combination of streams of SFQ pulses and in the temporal domain. In this work, we extend U-SFQ to introduce novel building blocks such as a multiplier and an accumulator. These blocks reduce area and power consumption by 2 \(\times\) and 4 \(\times\) compared with previously-proposed U-SFQ building blocks, and yield at least 97% area savings compared with binary approaches. Using these multiplier and adder, we propose a U-SFQ Convolutional Neural Network (CNN) hardware accelerator capable of comparable peak performance with state-of-the-art superconducting binary approach (B-SFQ) in 32 \(\times\) less area. CNNs can operate with 5-8 bits of resolution with no significant degradation in classification accuracy. For 5 bits of resolution, our proposed accelerator yields 5 \(\times\) -63 \(\times\) better performance than CMOS and 15 \(\times\) -173 \(\times\) better area efficiency than B-SFQ.

DANSEN: Database Acceleration on Native Computational Storage by Exploiting NDP

This paper introduces DANSEN , the hardware accelerator component for neoDBMS, a full-stack computational storage system designed to manage on-device execution of database queries/transactions as a Near-Data Processing (NDP)-operation. The proposed system enables Database Management Systems (DBMS) to offload NDP-operations to the storage while maintaining control over data through a native storage interface . DANSEN provides an NDP-engine that enables DBMS to perform both low-level database tasks, such as performing database administration, as well as high-level tasks like executing SQL, on the smart storage device while observing the DBMS concurrency control. Furthermore, DANSEN enables the incorporation of custom accelerators as an NDP-operation, e.g., to perform hardware-accelerated ML inference directly on the stored data. We built the DANSEN storage prototype and interface on an Ultrascale+HBM FPGA and fully integrated it with PostgreSQL 12. Experimental results demonstrate that the proposed NDP approach outperforms software-only PostgreSQL using a fast off-the-shelf NVMe drive, and significantly improves the end-to-end execution time of an aggregation operation (similar to Q6 from CH-benCHmark, 150 million records) by ≈ 10.6 ×. The versatility of the proposed approach is also validated by integrating a compute-intensive data analytics application with multi-row results, outperforming PostgreSQL by ≈ 1.5 ×.

Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference

Recent advancements in large language models (LLMs) boasting billions of parameters have generated a significant demand for efficient deployment in inference workloads. While hardware accelerators for Transformer-based models have been extensively studied, the majority of existing approaches rely on temporal architectures that reuse hardware units for different network layers and operators. However, these methods often encounter challenges in achieving low latency due to considerable memory access overhead. This paper investigates the feasibility and potential of model-specific spatial acceleration for LLM inference on FPGAs. Our approach involves the specialization of distinct hardware units for specific operators or layers, facilitating direct communication between them through a dataflow architecture while minimizing off-chip memory accesses. We introduce a comprehensive analytical model for estimating the performance of a spatial LLM accelerator, taking into account the on-chip compute and memory resources available on an FPGA. This model can be extended to multi-FPGA settings for distributed inference. Through our analysis, we can identify the most effective parallelization and buffering schemes for the accelerator and, crucially, determine the scenarios in which FPGA-based spatial acceleration can outperform its GPU-based counterpart. To enable more productive implementations of an LLM model on FPGAs, we further provide a library of high-level synthesis (HLS) kernels that are composable and reusable. This library will be made available as open-source. To validate the effectiveness of both our analytical model and HLS library, we have implemented BERT and GPT2 on an AMD Xilinx Alveo U280 FPGA device. Experimental results demonstrate our approach can achieve up to 13.4 × speedup when compared to previous FPGA-based accelerators for the BERT model. For GPT generative inference, we attain a 2.2 × speedup compared to DFX, an FPGA overlay, in the prefill stage, while achieving a 1.9 × speedup and a 5.7 × improvement in energy efficiency compared to the NVIDIA A100 GPU in the decode stage.

Implementation of Lightweight Convolutional Neural Networks with an Early Exit Mechanism Utilizing 40 nm CMOS Process for Fire Detection in Unmanned Aerial Vehicles.

The advancement of unmanned aerial vehicles (UAVs) enables early detection of numerous disasters. Efforts have been made to automate the monitoring of data from UAVs, with machine learning methods recently attracting significant interest. These solutions often face challenges with high computational costs and energy usage. Conventionally, data from UAVs are processed using cloud computing, where they are sent to the cloud for analysis. However, this method might not meet the real-time needs of disaster relief scenarios. In contrast, edge computing provides real-time processing at the site but still struggles with computational and energy efficiency issues. To overcome these obstacles and enhance resource utilization, this paper presents a convolutional neural network (CNN) model with an early exit mechanism designed for fire detection in UAVs. This model is implemented using TSMC 40 nm CMOS technology, which aids in hardware acceleration. Notably, the neural network has a modest parameter count of 11.2 k. In the hardware computation part, the CNN circuit completes fire detection in approximately 230,000 cycles. Power-gating techniques are also used to turn off inactive memory, contributing to reduced power consumption. The experimental results show that this neural network reaches a maximum accuracy of 81.49% in the hardware implementation stage. After automatic layout and routing, the CNN hardware accelerator can operate at 300 MHz, consuming 117 mW of power.

Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms

This perspective paper explores challenges associated with online crowdsourced data collection, particularly focusing on longitudinal tasks with time-sensitive outcomes like response latencies. Based on our research, we identify two significant sources of bias: technical shortcomings such as low, variable frame rates, and human factors, contributing to high attrition rates. We explored potential solutions to these problems, such as enforcing hardware acceleration and defining study-specific frame rate thresholds, as well as pre-screening participants and monitoring hardware performance and task engagement over each experimental session. With this discussion, we intend to provide recommendations on how to improve the quality and reliability of data collected via online crowdsourced platforms and emphasize the need for researchers to be cognizant of potential pitfalls in online research.

Data-driven modeling of reconfigurable multi-accelerator systems under dynamic workloads

Reconfigurable multi-accelerator systems used as computing offloading platforms in edge-cloud continuum scenarios usually have to deal with highly dynamic workloads and operating conditions. In order to properly take advantage of their parallel processing capabilities and increase execution performance for a given workload, these systems need to continuously adapt their configuration (i.e., number and type of accelerators) at run time. When working at the edge, additional requirements such as energy efficiency must be also met. In this paper, Machine Learning techniques are applied to extract predictive models of the execution of different combinations of hardware accelerators on a reconfigurable multi-accelerator platform, aiming at satisfying the previously mentioned continuous optimization needs. One of the key benefits of the proposed approach is that its data-driven models can transparently estimate the impact of the complex interactions between hardware accelerators due to run-time resource contention among them and with the rest of the system, as opposed to traditional modeling approaches that cannot include that information in an easy and scalable way (e.g., analytical models). The proposed models are complemented with a complete infrastructure to generate, execute and monitor dynamic workloads in FPGA-based systems. This infrastructure has been used to (i) quantitatively analyze resource contention in reconfigurable multi-accelerator systems and (ii) produce the training and evaluation datasets for the ML-based models using annotated power consumption and execution performance traces. Experimental results obtained with a reconfigurable multi-accelerator platform based on the ARTICo3 framework running the MachSuite benchmarks show that the proposed modeling approach is highly effective, with a relative prediction error of less than 5% on average for both power consumption and execution performance. Result also show that the ML-based models achieve high accuracy levels when predicting the impact of resource contention and accelerator interaction on both metrics, with a mean relative prediction error of less than 0.6% and a standard deviation below 4.7% for the worst case.

Implementation of FPGA-based Accelerator for Convolutional Neural Networks

This research paper presents a novel FPGA-based accelerator tailored for Convolutional Neural Networks (CNNs), specifically implemented on the Virtex-7 evaluation kit. By harnessing the inherent parallel processing capabilities of FPGAs, the architecture of the accelerator is meticulously crafted using Verilog. The FPGA implementation demonstrates a resource-efficient design, making use of 588 Look-Up Tables (LUTs) and 353 Flip Flops. Notably, the efficient utilization of these resources signifies a careful balance between computational efficiency and the available FPGA resources. This research significantly contributes to the field of hardware acceleration for CNNs by offering an optimized solution for high-performance deep learning applications. The presented architecture serves as a promising foundation for future advancements in FPGA-based accelerators, providing valuable insights for researchers and engineers working in the domain of hardware optimization for Convolutional Neural Networks.

Systematic analysis of FPGA-based hardware accelerators for convolutional neural networks

In the modern era, machine learning stands as a pivotal component of artificial intelligence, exerting a profound impact on various domains. This article delineates a methodology for designing and applying Field Programmable Gate Array (FPGA) based hardware accelerators for convolutional neural networks (CNNs). Initially, this paper introduces CNNs, a subset of deep learning techniques, and underscore their pivotal role in artificial intelligence, spanning domains such as image recognition, speech processing, and natural language understanding. Subsequently, we delve into the intricacies of FPGA, an adaptable logic device characterized by high integration and versatility, elucidating our approach to creating a hardware accelerator tailored for CNNs on the FPGA platform. To enhance computational efficiency, we employ technical strategies like dual cache structures, loop unrolling, and loop tiling for accelerating the convolutional layers. Finally, through empirical experiments employing YOLOv2, and validate the efficacy and superiority of our designed hardware accelerator model. This paper anticipates that in the forthcoming years, the methodology and research into FPGA-based CNN hardware accelerators will yield even more substantial contributions, propelling the advancement and widespread adoption of deep learning technology.

Accelerating autonomous vehicles: Harnessing FPGA power for deep learning advancements

In the rapidly developing field of autonomous vehicles (AV), the integration of deep learning algorithms has become the cornerstone for improving vehicle perception and decision-making ability. This study investigates the potential of field-programmable gate arrays (FPGAs) as hardware accelerators for deep-learning tasks in self-driving cars. This study used a Zynq UltraScale+ MPSoC FPGA board from Xilinx, known for its performance and adaptability, and combined it with a high-resolution camera to simulate real-world visual data encountered by AVs. The approach involved implementing a convolutional neural network (CNN) to perform tasks such as object detection, lane detection, and traffic sign classification. The model was quantized and converted to VHDL code using Xilinx Vivado HLS to optimize the deep learning algorithm for FPGA.The results show that the FPGA-based model significantly outperforms the traditional CPU-based model in terms of processing speed and energy efficiency without sacrificing accuracy. This study highlights the critical role of FPGAs in supporting deep learning tasks in self-driving cars and paves the way for safer and more efficient transportation solutions in the future.

Post-Quantum Signatures on RISC-V with Hardware Acceleration

CRYSTALS-Dilithium and Falcon are digital signature algorithms based on cryptographic lattices, which are considered secure even if large-scale quantum computers will be able to break conventional public-key cryptography. Both schemes have been selected for standardization in the NIST Post-Quantum competition. In this work, we present a RISC-V HW/SW codesign that aims to combine the advantages of software and hardware implementations, i.e., flexibility and performance. It shows the use of flexible hardware accelerators, which have been previously used for Public-Key Encryption (PKE) and Key-Encapsulation Mechanism (KEM), for Post-Quantum signatures. It is optimized for Dilithium as a generic signature scheme but also accelerates applications that require fast verification of Falcon’s compact signatures. We provide a comparison with previous works showing that for Dilithium and Falcon, cycle counts are significantly reduced, such that our design is faster than previous software implementations or other HW/SW codesigns. In addition to that, we present a compact Globalfoundries 22nm ASIC design that runs at 800 MHz. By using hardware acceleration, energy consumption for Dilithium is reduced by up to 92.2%, and up to 67.5% for Falcon’s signature verification.

1-Selector 1-Memristor Configuration with Multifunctional a-IGZO Memristive Devices Fabricated at Room Temperature.

Serving as neuromorphic hardware accelerators, memristors play a crucial role in large-scale neuromorphic computing. Herein, two-terminal memristors utilizing amorphous indium-gallium-zinc oxide (a-IGZO) are fabricated through room-temperature sputtering. The electrical characteristics of these memristors are effectively modulated by varying the oxygen flow during the deposition process. The optimized a-IGZO memristor, fabricated under 3 sccm oxygen flow, presents a 5 × 103 ratio between its high- and low-resistance states, which can be maintained over 1 × 104 s with minimal degradation. Meanwhile, desirable properties such as electroforming-free and self-compliance, crucial for low-energy consumption, are also obtained in the a-IGZO memristor. Moreover, analog conductance switching is observed, demonstrating an interface-type behavior, as evidenced by its device-size-dependent performance. The coexistence of negative differential resistance with analog switching is attributed to the migration of oxygen vacancies and the trapping/detrapping of charges. Furthermore, the device demonstrates optical storage capabilities by exploiting the optical properties of a-IGZO, which can stably operate for up to 50 sweep cycles. Various synaptic functions have been demonstrated, including paired-pulse facilitation and spike-timing-dependent plasticity. These functionalities contribute to a simulated recognition accuracy of 90% for handwritten digits. Importantly, a one-selector one-memristor (1S1M) architecture is successfully constructed at room temperature by integrating a-IGZO memristor on a TaOx-based selector. This architecture exhibits a 107 on/off ratio, demonstrating its potential to suppress sneak currents among adjacent units in a memristor crossbar.

Revolutionizing machine learning: A comprehensive analysis of ASIC hardware accelerators and their applications

The growth of the web over the past few years has led to tremendous data growth, which has provided a powerful impetus for artificial intelligence and machine learning. Machine learning algorithms are widely used in various classification and prediction problems. However, with the rich data types and needs, the traditional software computing method that relies on CPU can no longer meet the application scenarios under different requirements. Machine Learning (ML) hardware accelerators, especially Application Specific Integrated Circuits (ASICs), have become trendy to meet a variety of needs. This paper reviews the research on ML ASIC in the past few years, reviews the development of ML and ASIC design, and summarizes the characteristics of their use. It is followed by examples of various scenarios for which it can be used, such as medical diagnostics and internet of thing (IOT) terminals. Finally, the author analyses the existing problems and limitations, and gives the improvement methods of hardware and algorithm to deal with the related obstacles. It is hoped that this paper can provide some help and convenience for the subsequent related research.

FPGA acceleration of convolutional neural networks: Current applications and future prospects

In the contemporary era of big data, the utilization of deep learning technology has become widespread in the extraction of valuable analytical insights from the data under consideration. This technology finds extensive applications in various domains, including image information recognition, speech processing, and text language analysis. When it comes to accelerating convolutional neural networks (CNNs), the utilization of Field-Programmable Gate Arrays (FPGAs) offers distinct advantages over other hardware accelerators. Nevertheless, it is important to acknowledge that FPGA-based acceleration also comes with its own structural limitations. This article places its focus on two main aspects: firstly, it delves into the present-day application landscape and the latest developmental trajectories of convolutional neural networks. Secondly, it elucidates the inherent characteristics of FPGA implementations of CNNs. Furthermore, the article conducts a comprehensive examination of the pertinent constraints associated with FPGA-accelerated deep learning algorithms. The discussion extends beyond the present moment and ventures into the future, offering insights into the potential advancements in the field of deep learning. Importantly, it also brings into focus the prospective avenues for further research regarding the application of FPGAs in the domain of convolutional neural networks.

A comparative study of FPGA and CGRA technologies in hardware acceleration for deep learning

With the rapid growth of deep neural networks, their increasing demand for computing resources poses significant challenges to resource-constrained edge devices. Field Programmable Gate Array (FPGA) and Coarse-Grained Reconfigurable Arrays (CGRA) have emerged as promising solutions due to their parallelism and energy efficiency advantages. This paper conducts a comprehensive comparative analysis of FPGA and CGRA for accelerating deep learning workloads. It begins by introducing convolutional neural network architecture and optimization techniques. Various dataflow strategies are discussed for memory access optimization. The paper then analyzes representative cases to clarify FPGA and CGRA architectural advantages, along with optimization techniques through different dataflow strategies. The comparison reveals that FPGA offers higher throughput with fine-grained programmability for flexible bit-level operations and dataflow control. In contrast, CGRA excels in energy efficiency and area consumption with coarse-grained custom processing units and optimized on-chip interconnection networks. Quantitative comparisons are provided for throughput, energy efficiency, and area occupation. These insights can serve as a reference for selecting the appropriate architecture when designing custom hardware accelerators for deep learning workloads.

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Recurrent Neural Networks (RNNs) are useful in temporal sequence tasks. However, training RNNs involves dense matrix multiplications which require hardware that can support a large number of arithmetic operations and memory accesses. Implementing online training of RNNs on the edge calls for optimized algorithms for an efficient deployment on hardware. Inspired by the spiking neuron model, the Delta RNN exploits temporal sparsity during inference by skipping over the update of hidden states from those inactivated neurons whose change of activation across two timesteps is below a defined threshold. This work describes a training algorithm for Delta RNNs that exploits temporal sparsity in the backward propagation phase to reduce computational requirements for training on the edge. Due to the symmetric computation graphs of forward and backward propagation during training, the gradient computation of inactivated neurons can be skipped. Results show a reduction of ∼80% in matrix operations for training a 56k parameter Delta LSTM on the Fluent Speech Commands dataset with negligible accuracy loss. Logic simulations of a hardware accelerator designed for the training algorithm show 2-10X speedup in matrix computations for an activation sparsity range of 50%-90%. Additionally, we show that the proposed Delta RNN training will be useful for online incremental learning on edge devices with limited computing resources.

Revolutionizing machine learning: Harnessing hardware accelerators for enhanced AI efficiency

In recent decades, the field of Artificial Intelligence (AI) has undergone a remarkable evolution, with machine learning emerging as a pivotal subdomain. This transformation has led to increasingly complex algorithms and soaring data volumes, necessitating robust computational resources. Conventional central processing units (CPUs) are struggling to meet the demanding requirements of modern AI applications. In response to this computational challenge, a new generation of hardware accelerators has been developed to enhance the processing and learning capabilities of machine learning systems. Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), and Application Specific Integrated Circuits (ASICs) are among the specialized accelerators that have emerged. These hardware accelerators have proven instrumental in significantly improving the efficiency of machine learning tasks. This paper provides a comprehensive exploration of these hardware accelerators, offering insights into their design, functionality, and applications. Moreover, it examines their role in empowering machine learning processes and discusses their potential impact on the future of AI. By addressing current trends and anticipated challenges, this paper contributes to a deeper understanding of the dynamic landscape of hardware acceleration in the context of machine learning research and development.